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Abstract— Food markets are increasingly demanding the implementation 
of good agricultural practices programs (GAP) in the public or private 
sectors as a way to guarantee the sustainable and responsible production 
of safe food. Due to the large number of GAP programs being 
implemented, producers are often required to participate in several of 
them to comply with the demands coming from diverse buyers in different 
target markets; as a result, even though the majority of certificate 
requirements share factors in common, the costs of implementation and 
evaluation increase. In this context, a tool was created to analyze and 
manage multidimensional risks in agriculture (4DGAP tool) (evaluation 
of the GAP in four dimensions), developed through an alliance between 
Embrapa and IICA proposing methodological bases that would support 
the preparation and updating of indicators linked to the GAP programs, 
facilitate interplay between the different certification programs and 
likewise between programs and the producers, agribusinesses and 
governmental agencies that use them. In addition, its objective is to 
contribute to the reorganization of all kinds of rural farms, based on a 
concept of property planning in keeping with the technical and 
environmental parameters needed to comply with the principles of 
sustainable development. 





I. INTRODUCTION 


Since the end of the 90’s, GAP programs have become 
common and even mandatory in the food production 
process, especially in relation to food destined for fresh 
consumption (Amekawa 2009, Mattos et al. 2009). Over 
time, it has become necessary to understand the reasons for 
the success and failure of these programs so their 
effectiveness can be improved. (Srisopaporn et al. 2015). 
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A lack of specific tools to evaluate and monitor GAP 
programs motivate the need to find options that serve as a 
basis for their development. In some cases, simple 
monetization of the systems’ results has been used 
(Mandarino et al. 2019); however, the programs contain 
numerous variables that hinder a complete financial 
assessment. 


With few exceptions, GAP programs are based on the 
triad of food safety, worker safety (or sustainability of the 
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activity) and environmental protection, which has 
gradually evolved to resemble the makings of a system to 
assess the impact and environmental management of rural 
farms (da Cruz et al. 2006, Sabbag 2008, Blasi et al. 2016). 


Although in the majority of cases the use of indicators 
to evaluate sustainability generates an incomplete 
description of a very specific problem, when applied to 
highly subjective matters, such as certification of 
agricultural quality, it can become a valuable tool to 
reduce the realm of speculation (Binder et al. 2010, Coteur 
et al. 2016). Mendoza and Prabhu (2003) recommend 
using these indicators, since they make it possible for the 
different variables to interact in a holistic manner, 
including economic, environmental, biological and 
physical factors, hence allowing the condition of the 
agroecosystem to be evaluated and described. 


Notwithstanding the above, the difficulty of 
implementing a system depends on the selection of a 
certain number of socioeconomic, biotic and abiotic 
parameters that will serve as points of verification: having 
too few elements can result in a certification program that 
is too weak for the interested target audience, while an 
excessive number of elements can decrease the number of 
producers who are willing to participate in the program 
(Girardin et al. 2000, de Figueirédo et al. 2010). Currently, 
there are several formulas to define the indicators, but even 
today the premise is that the indicator must be 
representative, and where possible, multidimensional 
(Bertocchi et al. 2016, de Olde et al. 2017). 


While a system of GAP and good environmental 
practices is made up of a series of indicators of varying 
dimensions with different levels of importance where 
necessary, (Walter and Stiitzel 2009), generally speaking 
evaluating compliance with good practices is carried out at 
a global level, without regard for the size of each 
dimension or the interrelationships between them, which 
makes way for its implementation across all components 
of the chain, from the supplier of inputs to the consumer at 
the point of sale (Amekawa 2009). Compilation and data 
management of all the components are crucial, since they 
form the base for tracking the system and ensuring the 
chain of production/manufacture and stewardship of the 
products and inputs. 


According to Rodrigues et al. (2003), Hayo et al. 
(2007) and Van Passel & Meul (2012), the typical 
restrictions associated with the use of indicators in 
environmental (or sustainability) analyses result from the 
lack of detailed information about choices made during the 
planning process, which is when methods of selection, 
compilation and data grouping are defined that will form 
the basis for the subsequent use of indicators. 
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Given the current low cost of compilation and data 
storage systems, these activities no longer pose an obstacle 
and have become key steps in conducting a thorough 
evaluation of certification programs and the structuring of 
management systems geared toward their improvement. 
Subjective bases for the data sets compiled over time have 
been abandoned, which allows the databases to be studied 
and analyzed, especially where it relates to rural farms. 
Therefore, as these databases grow, the systems of 
evaluation can also operate in a timely manner, assessing 
advances in the good practices program being 
implemented on the farm over time. 


Another feature of the good practices programs which 
facilitates adoption of these evaluation and management 
processes is that compliance evaluations are carried out 
based on simplified verification lists which contain 
elements that are structured in keeping with a descending 
standard and scientifically based ceiling values that require 
integral completion in a binary form (yes or no); this 
avoids the occurrence of non-parametric subjectivity that 
is based on the evaluator’s experience (Hayo and Van der 
Werf 2002, da Cruz et al. 2006). When techno- 
scientifically derived binary systems are used in evaluation 
processes involving verification, these binary systems are 
more restrictive compared to those that adopt partial 
compliance as an option, since by selecting the indicators 
and parameters that must be completed, the system will be 
satisfied only when it achieves full completion of the key 
points of the program, avoiding exchanges and 
interpretations that could affect the environment and go 
against the objectives proposed by the system creators. 


Some GAP evaluation systems can use a combined 
standard that allows some elements of partial compliance 
to be included; in this case, the elements of mandatory 
compliance are clearly delineated in the binary evaluation 
and likewise the elements of partial compliance for 
purposes of their improvement (Amekawa 2009), 
including when they are evaluated with a certain degree of 
subjectivity depending on the evaluator (percentage, 
descriptive and qualitative, among others). However, what 
will define their approval in the evaluation will be the full 
presence of mandatory elements in the system, which 
emphasizes the importance of binary evaluation in the 
verification list process. 


Due to the expansion of global trade, the difficulty of 
harmonizing several GAP certifications programs, 
international recognition of many private and public 
certification “seals” and the need to expand client base, 
producers end up contracting several certification 
programs so they can sell their products to various buyers 
in different countries. This situation drives up the cost of 
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GAP programs, as a result of the multiplication of 
compliance evaluation costs and the corresponding audits. 


There is no interface between the certification 
programs, so rural producers and the certifying entities 
find it difficult to understand that several certifications 
share many requirements in common for which 
compliance is needed. Therefore, it was necessary to 
design an integrated evaluation tool, based on simplified 
indicators, that would analyze government and private 
certification programs and show the requirement 
dimensions in which the producer with. It is within this 
context that the 4DGAP tool arose, aimed at evaluating 
multidimensional impacts and steering the direction of 
GAP and good environmental practices certification 
programs. 


The 4DGAP tool was developed through an alliance 
between Embrapa and IICA with the aim of establishing 
methodological bases to continue preparing and updating 
indicators that would facilitate the interface between 
different certification programs and producers, 
agribusinesses and government agencies. It not only 
facilitates assessing compliance with commercial demands 
for food safety, but also the progress made in reorganizing 
rural farms in each country in keeping with technical and 
environmental parameters required to meet the precepts of 
sustainable development. 


I. METHODOLOGY 


The 4DGAP tool consists of a matrix of questions and 
binary responses (yes and no), created on a spreadsheet 
(MS Office Excel, Linux, Google, etc) that uses the same 
matrix mechanism. In order to build the questionnaire, 
information (metadata) is taken from the checklists of the 
main certification programs available on the market 
(GlobalGAP, Produção Integrada, TESCO Nature’s 
Choice, BRC/GFSI, etc.), whose data originate from 
collections in the field; added to this is the interpretation of 
the origin of each verification element as needed, which 
transforms them into indicators (Amekawa 2009, Mattos et 
al. 2009). 


Initially, the verification items in these programs were 
analyzed individually, point by point, selecting those that 
had the greatest capacity to represent the multidimensional 
character of the production context and the safety of the 
end consumer. 


As a result, the 4DGAP tool was configured as a matrix 
with five columns ( [Axis Y] _145): the first column 
contains the indicators in the form of questions, just as 
they would appear on the questionnaire sheet, and the rest 
of the columns correspond to each of the following 
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dimensions: environmental care, worker/farmer safety, 
food safety and economics. The eleven groups of 
indicators were distributed along the length of the matrix 
(Axis X_14n) as shown in Fig. 1. 


Detail of the checklist 
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Fig.1: Format of the 4DGAP matrix, where the indicators 
are placed on the horizontal lines and the evaluated 
dimensions in the columns. Source: Diaz et al. (2017) 


According to Diaz et al. (2017), the groups of 
indicators selected following the guidelines of Hayo and 
Van der Werf (2002) were derived from analyzing the 
checklists of GAP certification programs and the matrixes 
of the previously evaluated impact evaluation, resulting in 
the following elements: a) farm history and management; 
b) propagation material; c) soil and substrate management; 
d) fertilization; e) water management; f) crop protection; 
g) animal presence on the farm; h) hygiene and health; i) 
transportation; j) waste management and polluting agents; 
and k) training. 


In each of these groups there should be at least one 
representative indicator occupying the matrix line (Axis 
X_1^n), without setting a maximum number as a limit, 
consistent with how it is represented in each dimension. 


Subsequently, the 4DGAP tool was tested for its 
representativeness and the validity of the selected 
dimensions and indicators, with the support of, and 
analysis by government experts from several countries, 
field technicians, rural producers and users of safe food 
production and environmental assessment programs 
through meetings, interviews, and applications of the 
system on a phased experimental basis over a period of 
three years. 


The data entry page of the questionnaire is found in the 
operative section of the 4DGAP matrix; it consists of a 
closed questionnaire where each line must be answered 
with a binary response: yes (1.0) o no (zero), according to 
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the user’s compliance with the questions presented in the 
checklist, which generated the indicators listed in Y_1, as 
shown in Fig. 2. 


Erpa MAO 


GAP indicators 





Checklists for compliance with GAP on the farms 
instructions: Fill in the number “1” with the answers given by the producer / technician 








History and management of the farm 


Arrangement of the physical space of the farm 
Does the producer have the capacity to read and interpret a map 
‘or sketch of the farm? 


Does the producer have at this time a map or sketch that enables 
him to visualize the farm: production areas, facilities, roads, water 
resources, forest, etc.? 


If the previous response is no, could the owner design or help to 
design a map or sketch of the farm that enables him to plan the 


use of the physical space? 


Does the farm have a risk assessment that shows that the 
Production site is suitable for production, and does it have a 
management plan to minimize the risks identified? 





uction Questionnaire Program worksheet Results displayed in graphics Management matr 


Fig.2: Partial sample of the data entry questionnaire, with 
instructions on how to complete the columns on the right. 
Source: Díaz et al., 2017. 


Therefore, the indicator assumes a value of zero when 
the location being analyzed does not comply with the 
element of the checklist, or it complies partially, and a 
value of 1.0 when it complies fully. This eliminates the 
difficulty of assessing disparate indicators, since there is 
no comparison between them, just the verification of full 
compliance. It is fitting to emphasize that each indicator 
must show compatibility and the consequent score with at 
least one or more dimensions under analysis in the 
columns on the program worksheet. Therefore, the sum of 
the cells in each horizontal line should range in value 
between 1 and 4: the higher the value, the less specific and 
more representative it becomes. 


The final results are shown on the results and graphics 
worksheet, where a numerical table is generated to show 
the results of the different indicators, together with a radial 
graph to show the overall verification of the analysis and 
the final score, as seen in Fig. 3 and 4. 
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Fig.3: Presentation of the output data in numerical and 
graphical form, with the corresponding quantitative axes 
for each group of indicators for the general overview. 
Source: Diaz et al., 2017. 
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Fig.4: Presentation of the scenario analysis result, and a 
general overview for each of the individually analyzed 
dimensions. Source: Díaz et al., 2017 


The scoring produced by the tool is organized in three 
stages: 


1. The numerical standardization of the sum of the 
columns in the dimensions for each group of indicators, 
where the sum of the values in the dimension column is 
divided by the maximum number of group dimension 
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indicators (Idg) contemplated in that dimension (Equation 
1), which allows for a normalized response to be generated 
between zero and 1.0. 


Idg = (n marked indicators representative of the dimension 
/ nT total indicators representative of the dimension in the 


group) .......... (1) 


2. The summation of the indicators, first on the Y 
axis, in the form of dimensions in each group, where, in a 
situation of perfect compliance, the index reaches 1.0 and 
nears 0 where non-compliance with the demands on the 
interviewees’ control lists is detected, and then on the X 
axis, where, if there is full compliance with the 
requirement of the indicator group, a score of 4.0 is 
obtained, which decreases when the elements in the 
control list under analysis are not fulfilled. 


3. The summary phase forms the basis of the farm’s 
planning and environmental management phase, where it 
indicates to the decision maker the areas that are weak and 
the extent of measures to be taken. 


4. Once the final sum of the columns and rows is 
obtained, a single numerical value is generated so that the 
general matrix score varies between 0.0 (zero) and 44.0 
(Equation 2). The tool is reset using this value, which is 
then divided by the total value (44) and multiplied by 100, 
which generates a positive impact value (PI%) in the form 
of a percentage where values closer to 100% represent 
environments with better socioeconomic and 
environmental conditions and a greater degree of 
compliance with the food safety requirements compared to 
scenarios with values closer to zero, where the positive 
impacts detected would be less. 


PI (%) = (2 Group indicators and dimension)/44*100 


This simplified value facilitates the analysis of the 
overall evolution, and broken down by areas, of the same 
farm over time, as well as the comparison between the 
analyses of different farms, validating efforts to improve 
the production system and monitoring the individual, 
group and regional process. Therefore, the tool could 
appear in three different scenarios: 


1. The maximum value of 100% would mean the 
“maximum positive impact” has been achieved and the 
sustainable development prerequisites have been fulfilled 
by applying all the suggested agricultural and 
environmental practices, hence it should stay productive 


2. A value of 0% (zero) would signify the worst 
case, classified as “no positive impact”; because of this the 
system would register a high level of environmental 
degradation, which means no actions were identified in the 
unit being analyzed that could generate positive impacts 
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and the grower will need to implement a recovery plan for 
the farm, a status which reflects in the system as “Apply 
environmental management”. 


3. Any intermediate value between the two values 
above would mean the grower must execute a recovery 
plan for the farm, classifying it with the status of “Apply 
environmental management”. 


The final item constitutes a guide to resolve the 
problems described in the analysis generated after the user 
completes the 4DGAP. It applies the principles of 
environmental management as it relates to the general 
surroundings of the farm, including the social, economic, 
productive and ecological dimensions present in the 
productive space. If the index generated by the 4DGAP is 
other than 100%, it means the analysis detects risks in at 
least one of the analyzed dimensions on the farm area 
being investigated, but they can be mitigated using 
measures based on the following problem-solving matrix: 
a) What was the initial problem? b) What initial measure 
can be taken? c) What are some of the problems that occur 
after taking the initial measure? d) What can be done to 
resolve the resulting problems and for how long? This 
process is shown in Fig. 5. 


MA® 


Empa 


GAP indicators 





B) Example of a management matrix for the actions of the producers 


Find this type of training or training 
in locations near to the farm and, if 
possible, between production 
cycles 


There ts no training available in 
that aspect of knowledge 
within the region 


Mobilize groups of producers and 
request training from the 
corresponding authorities or the 
association to which they belong 


The producer is mot The producer does not have 


able to read or financial resources to pay for 
interpret a map or training 


sketch of the farm 


Make an immediate r jest to the 
The producer cannot leave the A art 


property for reasons of logistics 
of lack of labor for carrying out 
the daily activities 


corresponding authorities or to the 
association to which they belong 
for training on the property or 
distance training, if possible 








t Management matrix - producers | Q 


Fig.5: Example of an analysis performed in the risk 
management matrix of the 4DGAP system performed on a 
situation found in an analyzed rural property. Source: 
Díaz et al., 2017. 


This matrix, which is based on similar risk analysis 
systems found in administrative and environmental 
processes applied to agriculture (Campos and Melo 2008, 
Garza-Reyes et al. 2018, Huber et al. 2018), is filled out 
freely, since this is the way in which users present the 
problem, realize it exists, dedicate time to study a solution 
and think of ways to resolve it while taking into account 
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their technical and economic limitations and seasonal 
constraints. 


Given that the problem-solving matrix is a practical 
exercise that is under development, it can change over time 
as problems are solved, as they are replaced by others and 
as more complex solutions arise than what were initially 
proposed. 


WW. APLICATION AND CONSIDERATIONS 


After its final structuring, the 4DGAP was evaluated in 
the field between 2016 and 2018 through its application 
among a group of producers participating in different 
private or public certification programs involving different 
crops in Brazil (Brasilia Qualidade no Campo, Programa 
de Alimentos Seguros y Produção Integrada/Brasil 
certificado) and Costa Rica (GlobalGAP and BPA-MAG), 
with subsequent interview, using a questionnaire with open 
answers, aS a way to test whether rural producers 
understood it, gauge its acceptance among them, and 
verify both the ease with which technicians trained in its 
use could implement it and the tool’s robustness in relation 
to different production and certification systems (see Table 
1). 

Table 1. Application of the 4DGAP tool by country, the 
certified quality program, the number of producers and 
type of product. 





Country GAP Program/ Number Type of 




















certification seal of product 
producers certified 
analyzed 
Brazil Brasilia Qualidade 1 Vegetables 
no Campo! 
Produção 2 Fruits/ 
Integrada/Brasil viticulture 
Certificado? 
Programa 20 Fruits/ 
Alimentos viticulture 
Seguros (PAS)? 
Costa GlobalGAP*/BPA- 1 Fruits 
Rica MAG 
BPA-MAG> 3 Vegetables 
BPA-MAG> 1 Strawberries, 
mulberries 





' Brasilia Qualidade no Campo is an official program of 
the Government of the Federal District of Brazil. 


> Produção Integrada and the Brasil Certificado seal 
constitute the Brazil government’s official GAP 
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certification system; it is administered by the Ministry of 
Agriculture and is recognized internationally. 


3 The “Programa Alimentos Seguros” and the PAS seal 
constitute a private certification program used by 
producers in Brazil; it is recognized by multinational 
companies that purchase their products. 


4 GlobalGAP is a private certification program used by 
producers worldwide and it is recognized by multinational 
companies that purchase their products. 


5 Program BPA-MAG is the Government of Costa Rica’s 
official GAP certification system; it is administered by the 
Ministry of Agriculture and Livestock (MAG). 


As the field surveyors were applied, the refinement of 
the 4DGAP was also made, which started to be considered 
adjusted from the lack of new demands on the part of 
technicians and producers. When applied by different 
people under different circumstances, the robustness of 
these evaluation systems proves interesting for decision- 
makers (Shackelford et al. 2019) whether it is the decision- 
maker is the rural producer who needs monitoring and 
direction to manage the particular circumstances of his 
farm; the certification program managers, who would have 
a clearer vision of the program’s evolution; or the public 
agencies that would obtain feedback both on their projects 
with the aim of supporting these programs or the 
environment, and on the management of resources and 
efforts in this regard. 


According to Mauchline et al (2012) and Coteur et al. 
(2016), in order to guarantee the robustness of the system 
after its launch, interviews to generate farm analyses were 
conducted among different people and different agencies, 
ranging from the producer who is directly involved in 
production and the company's technical assistant, to the 
company that receives and processes the product. 
Therefore, besides the 4DGAP team that created the tool, 
other technical advisors and extension officers were 
trained to apply the method and evaluate impressions 
concerning the advantages and challenges of implementing 
the tool, the interpretation of results and the guidance 
given to producers. 


The first impression of the effectiveness of the 4DGAP 
tool came from the group of developers who applied the 
tool to different certification programs: whether the 
programs were complex or simple, they did not encounter 
any problems in terms of adaptability. Since the indicators’ 
core principles were represented across the different 
programs and given that there were no unclear questions 
that would be subject to interpretation and therefore put 
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the evaluation system at risk, the questions were answered 
in an efficient manner. 


With regard to the programs that had the largest 
number of respondents, we made sure to send the 
interviews to trained technicians in order to obtained 
external input. According to the responses gleaned by 
these teams, it was verified that although users initially 
thought the questionnaire was long, once it was being 
applied it was possible to discuss and find solutions to 
many of the problems that were detected; later on, this 
would facilitate the work of technicians as they supported 
producers, turning it into a positive factor. 


The tool was considered useful for revealing problems 
that often were not considered or were not obvious to 
farmers, which allowed targeted work to be done after the 
planning phase. 


Another positive factor was the numerical visualization 
based on the final classification that resulted from the 
farm’s evaluation and the partial values that ensued from 
the analysis, which shows the exact areas in which the 
producer could attain maximum results and investment 
options to reach the goal. This opened up the opportunity 
to assess the need for significant financial investment to 
reduce the wait time or select and fulfill critical high 
impact factors with less resources and time. As a result, the 
producer was able to face head on the difficulties that 
came with environmental management in his area of 
production, since he had a better understanding and a 
better basis for making decisions. 


One advantage of the tool that was mentioned was the 
fact of having a group of producers who were using the 
same certification program, since this fostered an 
environment of internal competition to attain the 
qualifications endowed by the tool, making it possible to 
compare producers and generate the sense that improving 
is a requirement. 


Finally, something that caught the attention of the 
technicians who applied the questionnaires and who 
worked directly with the group of producers is that the 
model allowed them to have a temporal view of the 
process, as they started to follow the evolution of 
sustainability standards and environmental impact, as the 
producer met the demands generated in the check list and 
then organized in the management matrix generated by the 
producer. These developments were previously restricted 
to eventual descriptive reports, when required by the 
certification programs, otherwise information would be 
lost to the process. 


In addition, since 4DGAP tool facilitates continuous 
transformation, increasing or decreasing ratings both 
globally and at the level of indicators, it was easy for 
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farmers to implement planning and impact verification to 
the extent they were re-evaluated. 


IV. SAFETY FOOD AND PUBLIC POLICY 
IMPLICATIONS 


For local governments, the intrusion of foreign 
certification systems, private or not, usually focused on the 
export of local agricultural products, affects the 
organization of the official food security system in the 
country. 


The existence of a tool that allows assessing and 
juxtaposing the requirements of the different certification 
systems operating in a country or administrative region, 
allows the opening of discussions regarding a policy of 
equivalence between the different "quality seals", 
whenever they depart the common basis of assessment 
through BPAs. 


Likewise, if this tool has an advisory system for 
organizing the productive environment in the form of a 
matrix of risk analysis and management of the productive 
environment, it also facilitates the planning of agricultural 
policy for the region or by culture, since the bottlenecks 
for the implementation of good practice systems, generally 
required for products focused on exports, they can be 
planned based on real demands, avoiding unnecessary 
expenses with under or over dimensioning efforts, 
resources and manpower. 


Therefore, the 4DGAP system offers an organizational 
advantage to support policies to support the agricultural 
sector of a specific country, region or productive sector, 
whenever they need to discuss the possibility of adjusting 
the official and unofficial certification systems in force for 
the local situation, facilitating the work of the production 
chain, product buyers or governments. 


V. CONCLUSION 


The 4DGAP tool was designed to conduct an 
integrated evaluation of the various certification programs 
(official or private) available on the market. In the sample 
analyzed, neither developers nor technicians who were 
subsequently trained to implement it encountered any 
difficulties in understanding or applying it to the various 
programs. 


The model provides the ability to monitor the 
improvement in the sustainability of the certified 
production system over time, maintaining the record of 
actions performed in the past and the planning of those that 
will still be necessary to achieve the optimization of the 
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process, minimizing the impacts environmental, economic 
and social aspects of the analyzed production system. 


After the initial impact that the evaluation, and as 
interviews were being held, a positive, collaborative 
attitude ensued among them during the analysis and 
discussion for solutions. 


The annotation and graphics system helped the 
producers in visualizing the existing problems in the 
evaluated areas, as well as in the application of the 
environmental management matrix to solve the problems, 
with the support of technical advisors and extension staff, 
which can also be useful in a process of agricultural 
government planning in determining the bottlenecks of 
agricultural policy for the region or the productive chain 
analyzed. 


The 4DGAP tool can be useful for analyzing the 
effectiveness of GAP programs, specifying which aspects 
or indicators reveal strengths or weaknesses and, at the 
same time, can also be used to assess the efficiency of 
different programs, whether private or official, allowing 
the harmonization among them, as a form of agricultural 
policy, if necessary. 
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